Papers with ActivityNet Captions
WSLLN:Weakly Supervised Natural Language Localization Networks (D19-1)
Copied to clipboard
| Challenge: | Existing methods to learn correspondence between visual segments and texts require temporal coordinates for training, which leads to high costs of annotation. |
| Approach: | They propose weakly supervised language localization networks to detect events in untrimmed videos . they train with only video-sentence pairs without accessing to temporal locations of events . |
| Outcome: | Experiments on ActivityNet Captions and DiDeMo show that WSLLN performs state-of-the-art. |
DEBUG: A Dense Bottom-Up Grounding Approach for Natural Language Video Localization (D19-1)
Copied to clipboard
| Challenge: | Existing models for natural language video localization are top-down and bottom-up . however, both approaches suffer several limitations, leading to performance degradation . |
| Approach: | They propose a top-down approach for localizing a natural language description in a video sequence . they propose 'DEnse Bottom-Up Grounding' which uses the temporal boundaries of each video frame . |
| Outcome: | The proposed framework matches the speed of top-down models while surpassing the state-of-the-art models. |
Annotations Are Not All You Need: A Cross-modal Knowledge Transfer Network for Unsupervised Temporal Sentence Grounding (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing work on temporal sentence grounding rely on expensive video-query paired annotations . despite this, there are no ground-truth annotations in the current work . |
| Approach: | They propose to use paired video-query and segment boundary annotations to generate temporal sentence grounding without training. |
| Outcome: | The proposed model outperforms existing unsupervised methods and beats supervised ones on two challenging datasets. |
Generating Structured Pseudo Labels for Noise-resistant Zero-shot Video Sentence Localization (2023.acl-long)
Copied to clipboard
| Challenge: | Existing zero-shot pipelines generate event proposals and then generate a pseudo query for each event proposal. |
| Approach: | They propose a Structure-based Pseudo Label generation (SPL) that generates free-form interpretable pseudo queries before constructing query-dependent event proposals. |
| Outcome: | The proposed method learns with only video data without any annotation . it generates free-form interpretable pseudo queries before constructing query-dependent event proposals . |
Sparse Frame Grouping Network with Action Centered for Untrimmed Video Paragraph Captioning (2023.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for paragraph captioning videos without event ground truths generate one sentence for each event, but without event labels, it is difficult to locate the transitions between events and minimize repetition. |
| Approach: | They propose a module that dynamically groups event information with the help of action information for the entire video and excludes redundant frames within pre-defined clips. |
| Outcome: | The proposed module outperforms the state-of-the-art methods on all metrics. |